Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees

Deep Reinforcement Learning (DRL) has achieved impressive success in many applications. A key component of many DRL models is a neural network representing a Q function, to estimate the expected cumulative reward following a state-action pair. The Q function neural network contains a lot of implicit knowledge about the RL problems, but often remains unexamined and uninterpreted. To our knowledge, this work develops the first mimic learning framework for Q functions in DRL. We introduce Linear Model U-trees (LMUTs) to approximate neural network predictions. An LMUT is learned using a novel on-line algorithm that is well-suited for an active play setting, where the mimic learner observes an ongoing interaction between the neural net and the environment. Empirical evaluation shows that an LMUT mimics a Q function substantially better than five baseline methods. The transparent tree structure of an LMUT facilitates understanding the network's learned knowledge by analyzing feature influence, extracting rules, and highlighting the super-pixels in image inputs.Comment: This paper is accepted by ECML-PKDD 201

The devil is in the details: Genomics of transmissible cancers in Tasmanian devils

Cancer poses one of the greatest human health threats of our time. Fortunately, aside from a few rare cases of cancer transmission in immune-suppressed organ transplant recipients or a small number of transmission events from mother to fetus, cancers are not spread from human to human. However, transmissible cancers have been detected in vertebrate and invertebrate animals, sometimes with devastating effects. Four examples of transmissible cancers are now known: 1) canine transmissible venereal tumor (CTVT) in dogs, 2) a tumor in a laboratory population of Syrian hamsters that is no longer cultured, 3) infectious neoplasias in at least four species of bivalve mollusks, and 4) two independently derived transmissible cancers (devil facial tumor disease [DFTD]) in Tasmanian devils (Fig 1A and 1B). The etiologic agents of CTVT, the bivalve cancers, and DFTDare the transplants (allografts) of the neoplastic cells themselves, but the etiologic agent is unknown for the hamster tumor.The effects of these transmissible cancers on their respective host populations vary. CTVT is spread in dogs through sexual contact and is at least 11,000 years old, placing the timing of its origin close to that of the domestication of dogs. Although genomic analyses of the tumor suggest evasion of multiple components of the dog immune system, dogs most commonly survive and often show evidence of spontaneous tumor regression within a year of initial diagnosis. For the infectious bivalve neoplasias, which have existed for at least 40 years, population effects vary from enzootic infections with no noticeable effects on population sizes to evidence of a catastrophic population decline. In Tasmanian devils (Fig 1A), the first infectious tumor discovered (DFT1; Fig 1B) has spread across approximately 95% of the geographic range of Tasmanian devils since 1996 (Fig 1C). DFTD is almost always fatal (Fig 1B), with >90% declines in infected localities and an overall species-wide decline exceeding 80%. Transmission dynamics appear consistent with frequency dependence, with DFTD spread by biting during social interactions, resulting in predictions of extinction from standard epidemiological models. Despite these predictions, long-infected devil populations persist at reduced densities, suggesting that individual-level variability in fecundity and tumor growth rate in infected individuals are key for understanding epidemiological dynamics. Additionally, the origin of the second, independent lineage of DFTD (i.e., DFT2) within 20 years of the discovery of DFT1 suggests that transmissible cancers may be a recurring part of the Tasmanian devils' evolutionary history, without causing extinction

Why are eligible patients not prescribed aspirin in primary care? A qualitative study indicating measures for improvement

BACKGROUND: Despite evidence-based guidelines, aspirin prescribing for the secondary prevention of stroke is sub-optimal. Little is known about why general practitioners do not prescribe aspirin to indicated patients. We sought to identify and describe factors that lead general practitioners (GPs) not to prescribe aspirin to eligible stroke patients. This was the first stage of a study exploring the need for and means of improving levels of appropriate aspirin prescribing. METHOD: Qualitative interviews with 15 GPs in the West Midlands. RESULTS: Initially, many GPs did not regard their prescribing as difficult or sub-optimal. However on reflection, they gave several reasons that lead to them not prescribing aspirin for eligible patients or being uncertain. These include: difficulties in applying generic guidelines to individuals presenting in consultations, patient resistance to taking aspirin, the prioritisation of other issues in a time constrained consultation and problems in reviewing the medication of existing stroke patients. CONCLUSION: In order to improve levels of appropriate aspirin prescribing, the nature and presentation risk information available to GPs and patients must be improved. GPs need support in assessing the risks and benefits of prescribing for patients with combinations of complicating risk factors, while means of facilitating improved GP-patient dialogue are required to help address patient uncertainty. A decision analysis based support system is one option. Decision analysis could synthesise current evidence and identify risk data for a range of patient profiles commonly presenting in primary care. These data could then be incorporated into a user-friendly computerised decision support system to help facilitate improved GP-patient communication. Measures of optimum prescribing based upon aggregated prescribing data must be interpreted with caution. It is not possible to assess whether low levels of prescribing reflect appropriate or inappropriate use of aspirin in specific patients where concordance between the GP and the patient is practised

Gomesin peptides prevent proliferation and lead to the cell death of devil facial tumour disease cells.

The Tasmanian devil faces extinction due to devil facial tumour disease (DFTD), a highly transmittable clonal form of cancer without available treatment. In this study, we report the cell-autonomous antiproliferative and cytotoxic activities exhibited by the spider peptide gomesin (AgGom) and gomesin-like homologue (HiGom) in DFTD cells. Mechanistically, both peptides caused a significant reduction at G0/G1 phase, in correlation with an augmented expression of the cell cycle inhibitory proteins p53, p27, p21, necrosis, exacerbated generation of reactive oxygen species and diminished mitochondrial membrane potential, all hallmarks of cellular stress. The screening of a novel panel of AgGom-analogues revealed that, unlike changes in the hydrophobicity and electrostatic surface, the cytotoxic potential of the gomesin analogues in DFTD cells lies on specific arginine substitutions in the eight and nine positions and alanine replacement in three, five and 12 positions. In conclusion, the evidence supports gomesin as a potential antiproliferative compound against DFTD disease

EnzyMiner: automatic identification of protein level mutations and their impact on target enzymes from PubMed abstracts

BACKGROUND: A better understanding of the mechanisms of an enzyme's functionality and stability, as well as knowledge and impact of mutations is crucial for researchers working with enzymes. Though, several of the enzymes' databases are currently available, scientific literature still remains at large for up-to-date source of learning the effects of a mutation on an enzyme. However, going through vast amounts of scientific documents to extract the information on desired mutation has always been a time consuming process. In this paper, therefore, we describe an unique method, termed as EnzyMiner, which automatically identifies the PubMed abstracts that contain information on the impact of a protein level mutation on the stability and/or the activity of a given enzyme. RESULTS: We present an automated system which identifies the abstracts that contain an amino-acid-level mutation and then classifies them according to the mutation's effect on the enzyme. In the case of mutation identification, MuGeX, an automated mutation-gene extraction system has an accuracy of 93.1% with a 91.5 F-measure. For impact analysis, document classification is performed to identify the abstracts that contain a change in enzyme's stability or activity resulting from the mutation. The system was trained on lipases and tested on amylases with an accuracy of 85%. CONCLUSION: EnzyMiner identifies the abstracts that contain a protein mutation for a given enzyme and checks whether the abstract is related to a disease with the help of information extraction and machine learning techniques. For disease related abstracts, the mutation list and direct links to the abstracts are retrieved from the system and displayed on the Web. For those abstracts that are related to non-diseases, in addition to having the mutation list, the abstracts are also categorized into two groups. These two groups determine whether the mutation has an effect on the enzyme's stability or functionality followed by displaying these on the web

Identification of histone modifications in biomedical text for supporting epigenomic research

Corinna K, Klinger R, Hofmann-Apitius M. Identification of Histone Modifications in Biomedical Text for Supporting Epigenomic Research. BMC Bioinformatics. 2009;10(Suppl 1):S28